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(54) Communication system with plurality of synchronised data links 



(57) A digital system (100) has two or more nodes 
(120, 130) and a communication channel (1 10, 1 11) for 
transferring a single stream of ordered data from one 
node to another. The communication channel (1 10) has 
a number of data links (110a-ll0g) for transferring a 
plurality of sub-streams of data in a parallel fashion in 
order to transfer more data than a single data link is 

: 100 



capable of transferring. Receivers (I32a-I32g) each 
have synchronizing circuitry (200, 202) for synchroniz- 
ing a byte dock and a frame pulse of each received data 
sub-stream to the byte clock and frame pulse of a prese- 
lected master one of the receivers such that inherent 
data skew is eliminated. 
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dacftr. £*. „ Z»* "!2 2 or TralTncT * Ti ctiaftnelUiat may be a con- 

serial manner owr «. mSSS^JS^LT? T °°™" f *"» s,resm B "«*"ed * > 

and a receiver, wherein the shgte il Z 22,?? 7* ° f *" HnkS "* havin 9 a ,ran * 

method divides the angle stream of daTa^nto a JuSJi f h I * ° ^ ha ™ 9 a p,urali * ° f The 
in each sub-stream of data. E?ch m!^*^££2£l "* 3 frame pulse periodical1 * 
of data links in a parallel manner TtoZ Z^l^ ST °™ 3 """P"*'* da * of the plurality 
delay time that may be dm^rZ TZ^Tf ? d3t3 sub - streams ' herein a first data link has a first 

a first received data SEES S.'J^JEXJ ! ^ ^ SUCh ^ 3 data Skew «« 
each received data sub-stream SS^J^S^S^ ?f r6Cepti0n ' 3 byte Clock is recovered from 

«« « frame 4^?,^^"?^. ** ,he *»™ wl$e 01 •» ™* '«*« "eta 
received dala sub-aream. M *" 5U: " 5 " ea '" • »ame synchronfced with me maaer 

S * cSi:SS"^ P*- 1" - ™s te r reeved dee, stream „ 

accompany,^ S^iSSST" ' C,eSen ' "" en,W "~ * • * <"* - "amp*. ,ha 

f phbp I T det3,led b,0Ck diagram of 3 data link weaver of Fig 2- 9 

FIGURE 8 .s a schematic illustrating interconnections of the plurality of data link receivers of Fig 2; 
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FIGURE 9 is a more detailed block diagram of the clock recovery circuit for the receiver of Fig 7- 
FIGURE 10 is a schematic of the bit shifter for the receiver of Fig 5- 

FirmP ]l ^ 3 f te d L a9ram WhiCh C ° ntr0lS ,he ^"ronization Process of the communication channel of Fia 2- 
FIGURE 12 is a timing diagram illustrating S2PSyncOut signal timing- 9 " 2 ' 

5 to'^a^ 

FIGURE 14 is a timing diagram illustrating bit rotation of the plurality of data links of Fig 2 to oroduce bvtP aiinn 
^ul^c 0 " 1 " 10 " W ° rd Cl0Ck ' accordin 9 t0 ™ as P ect <* Present invention; 9 * ^ a " 9n ' 
» anTspe^ 

FirnRP !«p 15 3 1! 0 * °l! art a ! Mng the pr0Ce$S ° f byte ali9nin 9 P' uralit y of data of Fig 2 - 

F GURF tr c » ° W ^ art UStrati " 9 the pr0CeSS 0< ,rame ^ronization used in each slave link 'of Fig z- and 

FIGURE 16C is a flow chart illustrating the process of frame synchronization used in the master .ink of Fig z. 

15 lie i.2caTed 0ndin9 nUmera ' S "* * ^ ^ ^ ^ re,er t0 responding P-ts unless 

[0012] Aspects of the present invention include methods and apparatus for transferring a single stream of ordered 
data over a commun.cat.on channel having a plurality of parallel data links which each may have ^aa™^ 
mherent delay In the following description, specific information is set forth to provkfe a Zou^ unSLTno oi the 

We " jt circuits and devices are m in *S™ *™ * tZSSZ Z 

descripton unnecessanly. Moreover, it will be apparent to one skilled in the art that specific details of ^blocks are 
not required in order to practice the present invention. 9 
[0013] FIGURE 1 1 is a block diagram of a portion of a digital system 1 00 illustrating a communication channel 1 10 
for transferring data between two nodes (120. 130) of the system, according to aspects of the pSsem tventl A 

SlS? m£ iTu 6r embod,ment ma * ™«P'« bidirectional data on the same physical media 

! ansmittin9 drcuitr y 121 • receivir >9 circuitry 122. and processing circuitry 123 Processino cir- 
TZ may include a digital signal processor, memory circuits, analog or linear circuits, or any oi a wSe ranoe o» 
known cr novel crcurts. Similarly, node 130 has transmitting circuitry 13? receiving circuitry 3 ^ and o^cesSno cir 
so curtry 1c 3. Phyacal media 1 15 connects transmitter 121 to receiver 132. and is twisted pSs of wire inThis SS?ment" 
Likewise physical media 116 connects transmitter 131 to receiver 122 embodiment. 

35 Sof FIGURE i liHSf f agra T n showin 9 more detai ' of the plurality of data links which interconnect the two 
"V* J I \ S T? d3ta ' mkS 1 10a " 1 1 °9 are illustrated, but communication channels with a greater numberor 

a TrlZT™ f 5" "H? emb0died ' aCC ° rdin9 10 aspects of the P resent w^on. C l^tTy 1 Sa-g dMdes 
m hh?!,h1T ° ^ W ° rd data °" 1 50 '"to a plurality of sub-streams of ordered byte data Data b2s 50 is 

40 Sthatal^^ 

halfn nh. 3 ! m ? „ ^streams are received approximately coincidental*. However, each data link 1 lOa-lTog 
time T, rT?' f ^ and ,he ,ran5,er delay time 0< one link is typically dtfferent from the transfer deS 

ZZ£^J?J^^ ™T CeS " ^ ^ linkS ' Therefore ' a Skew is induc * betweenThe ^ious 
45 co^^r fh u k I reC8IVerS 1 32a ' 1329 ' AcC ° rdin9 10 aspects of 1,16 P resent inven «°n, receivers 132a-132o 
foZa ^ I 17 . ^ ^ ' inkS S ° th3t CirCUitry 1 60 Can combine 1,16 P' uralit y ° f received data strelms to 
1 Lll. fTfoTvo ^ ° f ° rdered d3ta WhiCh is m bits wide - This si "9 le s 'ream of data can be bufS2 n 

S5S A h •'" T^" 56 10 3 C ° mm0n W ° rd d0Ck 171 befOTe bein9 sent to Passing circuitry 133 
[0018 A^rdingtoanasperfofthepresem^ andallofthe 

Siio F o^^^ 

TZtZ Z r . EaCh frame ' nClUdeS 3 ,rame pulse P° rtion 200 and a da< a Portion 210 The length 
of the frame is defined by a frame_count_width 220. and is typically 1024 bytes. Each byte cormrises a numbP^ST 

55 n ujt'ion T, by 3 datapath - wk « h variab,e - a 9i^en embodiment of digHa. system iST n?mb"? d "ta ^ 
55 1 1 0a-1 10g, the frame_count_variable and the datapath_width are selected when digital system IQoTZZn^ 1! 
ever, other embodiments may vary, as will be discussed later with reference to Tatte f 15 9 ' ^ 

EST.- « ram ' n9 Pulse and P 3 ^" forma1s are 9iven in Table 1. The framing pulse occupies two bytes and the 
least s-gnrficant two b.ts are masked and used to transfer status information. Framing 
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FRAME_SYNC_SEQ 201 . status bits FVALID 202 and FERF m n • 

characters are inserted in data portion 210 in «d*ii™ w 9 8 * nch,oni "«'on of channel 110/111 stuff 

achieve lock. The stuff characters shown in ST, ^XT*'!?!*? 6 ^ t0 a,,ow a clock reco very circuit to 
framing pulses and the stuff characters to eliminate the it S ni l*?? "^T hammin9 distance betw een the 
1 is zero filled lo occupy two byte positions * ***** ° f fa ' Se fram,n9 - Each stuff ch ^«er shown in Table 



DATAPATH. WIDTH 

4 

5 
6 
7 


Stuff Character* (hsxjy 
17 

IF | 

6F 

BF 


f Pattern 

p (FRAME SYNC crag ny 

4B 
2D8 
B62 


8 
9 
10 


63F 
8FF 
22FF 


2D8A 

F628 ~~ 
3D8B0 ~~ 
F62E0 ~~ 



Table 1: Framing Pul se Format 



[0021] Two status bits are defined FVALID 2n? inriir^ . .u 
2S end receiver failure) conveys the status of melc^Tr^SS !• **■ FERf ' 203 < far 

ter for use in enabling data transfer. S °° F (0Ul of frame ,nd,cator ) *> »» downstream rransmit- 

[0022] For example, during synchronization, receiver 132 s onF ica ^ 

pattern in the paytoad. Receiver 13*sOOFfet^SL?, 0F active and transmitter 121 isoutputting thestuff 
111. This is decoded in receiver 122 and lS£SEE£i ?TL I " ° (FERF) ° f the header *** " * ** 

For multiple date links, each link transmits °' a P ' urali * of date > ■* transmitters 400(i). 

[0026] For example, an eight bit bus DATA[7:0]: 



DATA'high = 7 
DATA'low = 0 
45 DATA'range = 7 downto 0 
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Pin Name 


lntamaU 
Exlsnuu* 


lnpnt/ 

Output 


Description 


RESET 


External 


Input 


Reset signal for framer and scrambler. Reset causes the scrambler to 
be preloaded with all Vi 


CLOCK 


External 


Output 


Byte clocks. These are identical clocks derived from the transmit bit 

clock. 


TMODE 


External 


Input 


Places the transmitter into test mode. 


PRBSEN 


External 


Input 


This signal may be used to enable/disable the scrambler 


NERF 


External 


Input 


Status signal indicating the status of the receiver on this channel at 
in is end oi trie unx. 


FERF 


External 


Input 


Status signal indicating the status of the receiver on this channel at 

>L. f a — +nA t*.t tka link 

tne tar end oi tne una. 


TFP1N 


Internal 


Input 


This signal is used by slave transmitters to align their frame pulse 

tvfln^mU m^j%*% at*j*a« tantlt t Ka m A/if «r 

cransmib se^uenc«B witn vqv atssver 


TFPOUT 


Internal 


Output 


This signal is output by the master during frame sequence insertion. 
It should be left unconnected on slave instances. 


TTLSIN 


Internal 


Input 


Transmit link status in. This signal is used to move ail the link 
transmit state machines into the SYNC state. 


TXSCIN 


Internal 


Input 


Transmit status chain in. This is a daisy-chained signal which is used 
to confer individual transmit status signals back to the master. 


TXSCOUT 


Internal 


Output 


See above. 


DATA.IN 


External 


Input 


((DATA PATHTiigh-1) downto 0) 
Data input valid when TDEN is TRUE. 


TDEN 


External 


Output 


This output defines the period when data may be read by the 
trsnsmitter and in designed to be interfaced directly to a FIFO. 


DATA.OUT 


Internal 


Output 


((DATAPATHTxigh-1) downto 0) 

Data output valid on rising edge of clock*. Connect directly to P2S. 



Table 2: Transmitter Interface Description 



[0027] Table 3 describes signals used by Frame Output circuitry 410 and Multiplexer 430. In response to a timing 
signal from the TX FSM the frame output circuitry 410 outputs framing bytes to the transmit multiplexer 430. Multiplexer 
430 then sends the n-brt wide data sub-stream to the parallel to serial (P2S) converter 450. P2S 450 sends the serial- 
ized data sub-stream to physical media interface 460. The output of interlace 460 is determined by the type of physical 
media 115 used in the communication channel. 

[0028] FIGURE 5 shows an implementation which fully synchronizes the bidirectional link 110/111 before transmit- 
ting data. The FERF signal that is conveyed on the master link is the only one used in synchronization. The other FERF 
inputs may be used to transfer proprietary in-band data. 

[0029] During the HUNT, PREBYTESYNC and PRESYNC states (described with reference to Fig 6) a fixed 
sequence is output in the frame payload which does not contain false framing patterns (see Table 1). At all other times 
multiplexer 430 forwards inputs from scrambler 420. All signals are active high unless explicitly stated otherwise. 
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15 



reset 
clock 
frame_sync_gate 

nerf 


input 
input 
input 

input 


synchronous reset 

internal clock 

Control signal used to define the period during 
which a framing pulse occurs 
Near end receiver failure 


prbs_en 
data-out 


input 
Output 


Scrambler enable 
((DATAPATrThigh-1) downto 0) 
data bus sent to receiver 


prbs_data_out 


input 


((DATAPATrThigh-1) downto 0) 
data bus from scrambler 



Table 3: Multiplexer Interface Description 
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ir T S? a ^ f ^ 10 9en9rate *• reference c,ock in an ■***•"•"» which uses 
dev.ce. Tabie 4 descnbes s,gnals used by scrambler 420. All signals are active high unless expficitly stated^erTe 



reset 



clock 



tmode 

Frame_sync_gate 



prbs_ena 



data in 



data-out 



input 



input 



input 



input 



input 



input bus 



output bus 



synchronous reset 



internal clock 



Test mode input 



Used to reset the MFSR 



Scrambling enable 



((DATAPATrThigh-1) downto 0) 
Input data 



((DATAPATHTiigh-1) downto 0) 
Output dat a (scrambled or descrambled) 



Table 4: Scrambler/Descrambler Interface Descripti 



ion 



SSil n.^ 3 " 113 'h r ^ nd0mi2es the data usi "9 a maximal length shift register (MLSR) with tap points selected to 

S e rlf m T blin » be S6,eCted aCCOrding 10 ? erceived ^tefion of the periodicity of the 

SeCc^ng ' ^ ^ aCCOrding to *• ■«*«■»" and may be frame or 

so Correlation and Periodicity 

S APf^ 0 -^ 0 " 1 s «» uence generated by an n-bit maximal length shift register is a binary seauence of 

* S3 
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Polynomial 


Reference 


x jr +x' a +i 


X31 X28 1 


x'+x b +i 


X7 X6 1 




k X43 1 


X W +X A<1 +1 


k X17 X14 1 



w 



Table 5: Scrambler/Descrambler Generator Polynomial 
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20 



[0036] During the framing sequence frame synchronous scrambler 420 is reset to all Vs. Scrambler 402 may be 
disabled permanently via the top level interface. When in test mode scrambler 420 ignores the data and outputs the 
scrambler code, descrambler 720 (see FIGURE 7) behaves normally. 

[0037] Transmit FSM (Finite State Machine) 440 outputs codes to enable the frame sequence. It also accepts a 
master framing pulse input to which output frame sequence timing is locked. The transmitter waits for removal of FERF 
before enabling normal operation of TXEN. In test mode input data is ignored and the frame period is set to 16 bytes. 
Signals used with Transmitter Finite State Machine 440 are described in Table 6. All signals are active high unless 
explicitly stated otherwise. 



25 



reset 


input 


synchronous reset 


clock 


input 


internal clock , 


tmode 


input 


Test mode input 


frame_sync_ext 


input 


for slave modules, this signal is connected to the 
tfpout port of the master. 


txjink_sync_in 


input 


transmit link status in. Enables the transmit state 
machines to move to PRESYNC state. 


ferf 


input 


Far end receiver failure indication for final link 
synchronisation. 


tx sync_chain_in 


input 


transmit status chain in. 


Tx_sync_chain_out 


output 


transmit status chain out. 
tx_sync_chainjn and tx__sync_chain_out are 
connected together for the master module. Each 
state ANDs the tx_sync_chain_in signal with its 
own status. 


frame sync_gate 


output 


active high for each for each frame period. 


CASTE 


generic integer 


program the module as a master or a slave. 



30 



35 



40 



Table 6: Transmit Finite State Machine Interface Description 



45 



[0038] Table 7 lists the states and codes of FSM 440. Signal Frame_sync_gate_int is an Internal framing pulse. Sig- 
nal Frame_sync_disparity is a difference between Frame_sync_gate_int and Frame_sync_jgate_ext. In the table, T 
so denotes a signal in its active state, '0' inactive. This may not correspond with high and low logic levels within the design. 
[0039] FIGURE 6 is a state diagram illustrating the operation of the data link transmitter 400. 
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20 




25 



Table 7: TX Finite State Machine Transition Table 
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\ Ittl T ?f SUt> ' Stream fr0m Ph/SiCal media 701 in P^ 03 ' media irt erface circuit 702. 
Recovers the data using an appropriate clock phase in clock recovery circuit 704 

• Converts the serial data into byte data with an appropriate byte clock in serial-to-parallel circuit 706 

• Recovers framing pulses in frame check circuit 710. »«rw»i'uo. 

• Recovers bit alignment from framing patterns in bit shifter 730 

• Descrambles the encoded data in descrambter 720. Descrambler 720 descrambles the received data sub stmam 
in a complimentary manner to scrambler 420 receivea aata sub-stream 

' p^nSSer'' * * differentia ' ^ ^ in ^ ™ e 750 in injunction with tap 

• Test circuit 760 provides methods for determining if receiver 700 is functioning properly. 

[0041] FIGURE 8 is a schematic illustrating interconnections of a plurality of data link receivers Toom M. .rm» 
receivers are cascaded to provide wKle data-paths and data rates highe'han L supper 2 by any o e 22 link £? 
theTh. 9 ,0 T* ° f ,he PreSent inven,i0n ' 0ne data ,ink receiver is designed as the mS.^ £SS££ 

5S «^ s C£C^tl^— ,0 receiver 700 as — - F « 7 — * «• » 
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Pin Name 


Int/ 
Ext 


Input/ 
Output 


Description 






Input 


Rpopfr ciimfll fnr rwpivp QPction 


CLOCK 


Ext 


Output 


Word clock. Derived from the transmit data and the local reference 


TMODE 


Ext 


Input 


This signal places the receiver in test mode 


TO A CC 




UllLpUl 


lDulCalco a Vcni pass. 


TFIN 


Ext 


Output 


Indicates test completion. 


RDVAL 


Ext 


Output 


I nifi OUipuc ucnuco wcieu umu lo VailU OH Ulc iHJtx lt\ y uia oiiu ma j 

be used to interface directly with a FIFO. 


S2PSYNC0UT 


Int 


Output 


lais signal is uses to onve me otroi hi^hm signals on aii siave 
instances, it snouia oe leu unconnecveo on bju uastaricea except uic 
master. , _ . 


RFPIN 


Int 




Input 


1 nis sign&J is UlScu to synchronize me uyve ujigixmeiii. uu oiavc 

instances. This should be tied FALSE on the master. 


BLCIN 


Int 


Input 


Byte Locked Chain In: Daisy chain which confers link byte lock 
status to the master „, _. 


LBLIN 


Int 


Input 


1 jIHK dVvG IjOCKCQ lu. MiUUQTOp Slgum unveu uy master uu uiujwim; 

the link is byte locked. 


LS1N 


Int 


Input 


| ^PK oync in! MUlviuXop signal anven oy ioc master to muica lc iuc 
link is fully synchronized. 


BLCOUT 


Int 


Output 


See BLCIN. 


LBLOUT 


Int 


Output 


See LBLIN. Connected on master only. 


LSOUT 


Int 


Output 


See LSIN. Connected on master only. 


RFPOUT 


lrt 


Output 


This signal is used to force alignment on the output bytes of all 
instances which form a link. It should be left unconnected on all 
instances except the master. 


OOF 


Ext 


Output 


These outputs indicate whether the respective receivers are in-frame 


FERF 


Ext/ 
Int 


Output 


Far end Receiver Failure. This signal is used to convey downstream 
receiver failures to upper layers and to synchronize the flow of data. 


PRBSEN 


Kxt 


Input 


This input may be used to disable the receive descrambler 


DATAJN 


Int 


Input 


((DATAPATrFhigh'l) downto 0) 

Received data from the S2P valid on the rising edge of WordClock. 


DATA_OUT 


Ext 


Output 


((DATAPATH-high-1) downto 0) 

Receive data valid on the rising edge of WordClock when RDVAL is 
asserted. _ , 



Table 8: Receiver Interface Description 



[0043] FIGURE 9 is a more detailed block diagram of the clock recovery circuit 704. Clock generator 900 generates 
a plurality of clock phase signals 902(i) based on reference clock 903 using known techniques. Each clock phase signal 
902(i) differs in phase by approximately 1/8 of a bit period. Clock recovery circuit 901 then compares edge transitions 
on received data sub-stream 904 to each of clock phase signals 902(i) and selects a clock phase signal that is most in 
correspondence with the received data sub-stream. In this manner, a bit clock is formed that has a fixed time relation to 
local reference clock 903. 

[0044] S2P circuit 706 then converts the serial received data sub-stream to a sequence of parallel data bytes on 
data bus 910. S2P 706 also forms byte clock on signal line 91 1 in response to the bit clock signal. When S2P circuit 706 
is instantiated during the integrated circuit design process, the parallel data byte width (n) is selected, as discussed with 
reference to Table 15. 

[0045] According to an aspect of the present invention, signal S2PSYNCOUT is provided by master receiver 700(0) 
and causes S2P circuit 706 to synchronize byte clock signal 91 1 with the byte clock signal of master receiver 700(0), 
as will be explained in more detail later. 

[0046] FIGURE 10 is a schematic of bit shifter circuitry 1000 that illustrates a byte data path width of five bits. Bit 
shifter circuitry 1000 provides a shiftable biMap point that is operable to align byte boundaries of the received data sub- 
stream with the by:e clock of the receiver. Latches 1001 and 1002 are cascaded so that together they buffer ten bits of 
sequential data. Multiplexer 1010 is operable to tap any five bits of data from latches 1001 and 1002 in response to 
select signal 1021 formed by counter 1020. The tap point is incremented in response to shiftjnc_ena signals received 
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70 



/5 




shift_inc_ena 
data_out 



input 



Input 



Output 



((DATAPATH-high-1) downto « 
data bus from S2P module 



one-bit shift e nable (HUNT sutenny^ 



auatt 

((DATAPATHTiigh-l) downto 0) 
data bus connected to the aWra mbler 



Table 9: Bit Shifter Interface Description 



20 



25 



30 



35 



40 



clock 



reset 



iput internal clock 



frame_sync_gate 



data_in 



out_of,frame 



fvalid 



ferf 
frame_sync_ rx 



input 



input 



input 



input 



output 



output 



output 



synchronous reset 



frame synchronisation pulse 



J w«i»«wn pyAiox; 

((DATAPATIThigh-1) downto 0) 

data bus from decbit shifter module 



receiver status signal 



frame valid signal 



far end receiver failure signal 



signal active when a framing pulse is :^giyerf_ 



Table 10: Frame Checker Interface Description 
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curtry so that the SSSI^SS tame ZrJ *2 * t0 * e frame *«* * 

This process will be described T ^f - ^ * the " nk - 

instantiated during design of an ' P,Pe " n6 " " hen 1,16 inteQrated ™ * 

Sing toaVS^eZ^ « * »"~ " b — *. data links, 

master shfts te £ poin as wShHats to! £E ' °' *? P ' P6line * 861 by FSM 74a 2n -PP"*** the 
through all possible ZeVZ^ * "~ * t1B *"* te CydeS 

fore proportional to the number of tib^^ZSl * ? aX,mUm lock time is there- 

frame Tync_out and -a^Tlo 7? ^ ^ M in,ormafo " 

[0050] Table 1 1 describes llwfj^ * 3nd al, 9 nment »e receive dafc, valid signal 
otherwise. * ^ "* byte pipe 750 - « si 9^ « active high unleus explicated 



10 



EP 0 996 262 A1 









reset 


input 


synchronous reset 


clock 


input 


internal clock 


data-in 


input 


[{UAi Ar Ai ri nign-ii down to uj 
data bus from descrambler module 


frame_sync_rx 


input 


a framing pulse is received 


rdval in 


input 


data valid when there is no framing pulse 


byte.sel 


input 


Selecter for the pipeline stage tap point. 


frame_sync_out 


output 


connected to RFPOUT top-level signal. This signal is 
used to force alignment on the output bytes of all slave 
instances of a link. 


fvalid 


input 


frame valid, used to gate rdval. 








rdval_out 


output 


ready data output 1 


data-out 


output bus 


received data valid 



Table 11: Byte Pipeline Interface Description 



[0051] When test mode is enabled, test circuitry 760 checks for set bits in the descrambied data stream. Any set 
bits found are erroneous when the transmitter is also in test mode. Test status is conveyed to the core via signals tpass 
and tfin and may also be read by the scan chain, not shown. Table 12 describes signals associated with test circuitry 
760. All signals are active high unless explicitly stated otherwise. 

[0052] A single test is initiated by asserting the TMODE pin on the transmitter and receiver. This test uses 1 6 byte 
frames to provide a faster method of achieving synchronization. Both transmitter and receiver must be placed into test 
mode as the transmitter must be inhibited from transferring user data. Once the system is synchronized an all zero 
frame is transferred over the channel and the descrambied results are checked for set bits. Any set bits will result in a 
test failure, which will be conveyed via TPASS. Resetting the TMODE pin will result in loss of synchronization and there- 
fore a delay before user data transfer may be resumed. 

[0053] Testing is done in conjunction with scan chains and may involve, for example, one of the following: 

Integration into an existing scan chain 
Integration into an additional scan chain 

Integration in the boundary scan chain indexed by a particular instruction code within an IEEE1 149. 1 implementa- 
tion. 

[0054] Communication channel 1 1 0/1 T1 supports both clocked scan and multiplexed flip flop test insertion. 









reset 


input 


Synchronous reset 


clock 


input 


Internal clock 


data jn 


input 


((DATAPATHTugh-l) downto 0) 
Data bus from S2P module 


ttnode 


input 


Enable test mode 


prbsen 


input 


PRBSEN recovered from the frame header 


tpass 


output 


Test passed 


tfin 


output 


Test phase complete 



Table 12: Built in Tester Interface Description 



[0055] Receive Finite State Machine 740 controls the operation of receiver 700. Table 13 describes signals associ- 
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ated with FSM 740. 



reset 
clock 
tmode 
Frame_sync rx 
extemaLframe_rx 


input 
input 
input 
input 
input 


- iigafflHggmiiMaHBBB 

synchronous reset 
internal clock 
Test mode input 
framing pulse is received 

A framing pulse is received from the master. Used 
to synchronize byte alignment on slave instances 


byte_locked_chain_in 
bytejocked chain out 


input 
output 


Enables master to determine when a link can 
transition from PRE BYTE SYNC to PRESYNC. 
Daisy chain which confers link byte lock status to 
the master. 


delta__courited_chain_in 
delta_counted chain out 


input 
output 


see above 

Enables masters to determine when link can 
transition from PRESYNC to SYNC. Daisy chain 
which confers link delta count status to the master 


link_byte_locked_in 
link_byte locked out 


input 


see above 

multidrop signal driven by the master to indicate 
the link is byte locked 


link_sync_in 
link_sync out 


output 
input 


see above. Connected on master only 

multidrop signal driven by the master to indicate 

uie iinK is iuny synchronized. 


byte_sel 
out_of frame 


output 
Output 

output 


see above. Connected on master only ~~ 

(UogJPIPELmE JVIDTHM downto 0) 
select one byte amonft the pipeline ones 


s2p_sync_out 

frame_sync_gate 
frame_sync valid_gate 


output 

output 
output 


indicates that the receiver is in or out of frame 
master sync output to ensure that S2P byte clocks 
are sufficiently aligned 

valid during the frame sync pulse. 


bit_shift_inc_ena 
post_ds_frame_sync__ 
valid_gate 


output 
input 


a frame penodic signal valid for a single clock cycle, 
shift enable for Bit Shifter Module 
Frame_sync_vahd_gate delayed by the latency of 
the descrambler. Used for byte alignment purposes. 


CASTE " f ' 


generic 
integer 


programs the module as a master or a slave. 



Table 13: Receiver Finite State Machine Interface Descripti 



[0056] FIGURE 1 1 is a state diagram which controls the synchronization process of receiver 700 The framino oat 

xsr x:iv:i s tt^t 400 is instantiated - as -p-" * »• fr-e-Sumc z^tZ^i 

[0057] Table 14 provides a detailed description of the state transitions of FSM 740. " 
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Table.U: Receiver FSM state table 



[0058] The various states illustrated by FIGURE 1 1 and Table 15 are as follows: 

HUNT (H): During this state the framing sequence is hunted on a bit-by-bit basis. Scrambling is disabled. When a 
correct framing sequence is identified the state changes to P REBYTESYNC. 

PREBYTESYNC (PB): During this period the state machine shifts the tap point of the byte pipeline until the framing 
sequences are aligned with the RFPIN (receive framing pulse —driven by the master). 

PRE SYNC (PS): During this period the frame header sequences are checked frame by frame. An incorrect framing 
sequence results in a return to the HUNT state. 6 correct frame sequences complete on all pairs will cause transi- 
tion to state SYNC. 5 is the max_delta_count_c variable specified when receiver 700 is instantiated, as described 
in Table 15. 

SYNC (S): The scrambler is enabled, a errored frame sequences will cause the state to return to HUNT, a is the 
MAX_ALPHA_COUNT_C variable specified when receiver 700 is instantiated, as described in Table 15. 

[0059] A counter in control circuitry 770 continually monitors progress through a frame. Two strobes are generated 
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which are used throughout the design for timing purposes, as follows: 



SSiJSf fe 3 Si9na ' WhiCh iS dUrin9 ,ramin9 PU ' SeS and iS USed t0 9a,e me ™W* ff0m » a taw 
Frame_sync_valid_gate is a signal which is active in the clock cycle immediately following the framing pulse. 

■ I0 S2 ,™=?? Chain • b/te l0Cked Ch3in ( blcin/b,co "t). k used to interconnect a plurality of receivers 700 as illustrate 

I S VT" ^ 5,376 behaVi ° r differs ,n ,he master 1,16 chain output SITES is on a 
the cha.n oulput represents the chain input ANDed with the its own status 

P^TESV^ «- S2PSYNCOUT, .ed in the 

Sff 1 h synch ™ 1lzation - according to aspects of the present invention, will now be described in detail Durina 
hnk synchron.zat.on the master receiver issues a pulse on signal S2PSyncOut which resynlnroSes Sale sS 

ST" ,S pTo S u D e c ,ha, 9 Si " 9le W ° rd d0Ck 1202 ™> be used t0 reliab| y clock all the S^TiSiX 
s,gnahs shown ,„ RQURE 12, illustrating S2PSyncOut signal timing. This resynchronization strobe , TaS whence 
master .s ,n the HUNT state, and is responsive to the word dock signal, as indicated at 1200 Wad clo* 1202* the 

T*** * ^ reC6iVer 7 °° (0) - ° Ue 10 bit s ^™**°" - -aster receiver SS.52 by2 the 
El fJSST ^ " d with the b * e c,ock «• ™ster «*e.Ver 700. as indicated at 1 201 . 
[0063] FIGURE 13 is a taming diagram illustrating byte clock synchronization of the plurality of data links of Fin 2 
according to an aspect of the present invention. Timing for only three links are illustrated u2£5 S3£ 

cTo* V300 1 S 2 TJ ZXZT* '"J ^r™^ B6,0re ' ink ^chronization. each o, ^SSSS 
o plh hi!f V ? asd f u fsed earl1 ^ Each link then forms a byte clock 1301. 1303 and 1305, but the timing 
ofeach byte clock s.gnal is not related to the others, as illustrated in FIGURE 13. Therefore. disadvantageoudyaS 
CM. cn not be rel.ably used ,o dock all of the received data sub-streams of digrtal system node 1 30 as ^n aTtime 

. ? f SCUSS6d S2PSYNC0UT signal 1310 is asserted during the HUNT state of the master and sent 
to the slave data recovers at tame 1320. This connection is shown in FIGURE 8. S2P 706(0 in each sl^eTeSv* 70o J 
■s operate to reset * own byte dock counter in response to the S2PSYCOUT signal 13 0 Sue 

2hS S 2P^ 

each S2P 706(i) places a new data byte on rts data_out bus 771 (i) in accordance with the realioned bvie clocks Tc 

SJ^r^S ^TTf T accordin9 10 an as * ect oi the -^T!Z£^X 

clock 1360 can be used to clock the data sub-streams of all of the receivers, as shown at time 1351 This sinale bvte 
tim?i^ 

[0065] However, the data bytes framed by the byte docks do not necessarily correspond to the underlyina data sub 

ZTs^oZ?^ ^ n ° W b69in 3t W 3 33 ShWn 31 137 °- * ^^S^bJS a bit 

[0066] FIGURE 14 is a timing diagram illustrating bit rotation of the plurality of data links of FIGURE 2 to oroduce 

TJSZSS 'ST* ^ ** 1360 ' aCC ° rding 10 a " aSpeCt 01 ,he present inventon * «SS ean eTthe 
brt tap po.m of bit softer crcurtry 730 in each receiver is set to align byte boundaries of the received date sub stream 

S SSSSS* !! Sh0Wn * U0 °- ' n R9Ure H SignalS 771 » are 1,16 link « b * e signals ^on Ss 771 of F c re 7 
and signals 772(.) are the rotated link(i) byte signals on bus 772 of Figure 7. Now advantageously Li ) byte Sin 
w,th brt 1 as shown at 1410 and link(2) bytes also begin with bit 1, as shown at 141 1 ' * ^ 

^ 'L k ^"u aCCOrdin9 10 an aspect of the present invention, each data link now provides a sub-stream of correctlv 
ordered bytes hat can be combined to form a single received data stream of ordered data 41 can SSjSS 
wrth a angle clock ,nto a single processing drcuit. such as single FIFO 170 of FIGURE 2 H^er ditent inh^em 
May. ,n each data link may result in byte skews between received data sub-streams 

iZroL ,S a . timin9 di39ram illustra « n 9 frame synchronization of the plurality of data links of Fig 2 in order 

SET ^Gut P^^r rGCeiVed ^ 3CC0rdin 9 to an ^ of *■ Preset tentaon 

hi . r w * UStrateS ° n,y three links for c,arit * a master link has a frame pulse 1510 at time 

SLY"? ^ k ( } ^ 3 frame PU,SS 1520 * time 1S02 ' and s,ave link < 2 > h as a frame pulse S at t m , iS? 
« 4f « ^ S d S? (0nS ^ ' r ° m "* Nnk) at time 1503 ' ,or exam P le - wi « « ^ corrSfly «SU 

S which 22 i 6 "?, ° GU " E 1 5A ' 3CCO,din9 ,0 3n aSpect 0f ,he P resent inventi0 "- reiver Xas a byS pipe 
750 wh.ch has several tap points, as discussed earlier. Five are illustrated (1511-1514). but other embodiment can 
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[0071 ] Likewise, referring to FIGURE 1 5B, a skew 1 550 of up to four bytes can be corrected with a pipe having five 
tap points. By selecting tap point 5 illustrated at 1535, and by selecting tap point 4 illustrated at 1514a, all three frame 
pluses 1510b. 1520b, and 1530b are aligned at time 1560. 

[0072] Therefore, as the length of byte pipe 750 is increased in each receiver, the amount of data link skew that can 
be tolerated is increased, according to an aspect of the present invention. 

[0073] FIGURES 1 6A-1 6C are f tow charts illustrating the process of synchronizing the byte clock and frame pulse 
of each received data sub-stream to the byte clock and frame pulse of the master received data sub-stream such that 
data skew is eliminated. FIGURE 16A is a flow chart illustrating the process of byte aligning a plurality of data links. In 
step 1600, a data transmission channel is instantiated using multiple data links. In step 1602,transmission is begun on 
each link pair with stuff characters in the payload and the framing pulses on all links synchronized. In step 1604, each 
receiver is synchronized to the data sub-stream using stuff characters to recover a bit clock relative to a receiver local 
reference dock. In step 1606, a byte clock is formed on each link relative to that link's bit clock. In step 1608, slave link 
byte clocks are synchronized to a master link byte clock. In step 1 61 0, a test is performed to determine if a frame pulse 
is being received on each link(i). If not, a bit tap point on link(i) is shifted to rotate data bytes until a frame pulse is rec- 
ognized. At this point, all links are now synchronized to a common word clock and the byteJocked_chain is asserted at 
step 1614, according to an aspect of the present invention. 

[0074] FIGURE 16B is a flow chart illustrating the process of frame synchronization used in each slave link. After 
each slave link has successfully byte aligned itself at step 1614, the master asserts signal LBLOUT and a test is per- 
formed by each slave(i) to determine if slave(i) is in frame sync with the master at step 1620. If not, at step 1622 the 
byte tap point of each slave(i) is shifted until frame synchronization is detected. Once each slave(i) is frame synchro- 
nized, it asserts its portion of the linkjbytejocked chain and waits in step 1624 for the master to assert signal LSOUT 
indicating all slave(i) are frame synchronized. Once this occurs, each slave data link begins to transmit payload data 
over the synchronized channel. 

[0075] FIGURE 16C is a flow chart illustrating the process of frame synchronization used in the master link. After 
all slave links are byte aligned at step 1614, the master link asserts the LBLOUT signal to start the frame synchroniza- 
tion process at step 1 640 and then waits two frames at step 1 642. The master then monitors the linkjbytejocked chain 
to determine if all slaves are in frame sync with master at step 1644. If not. then the master shifts its byte tap point one 
position at step 1646. waits two frames at step 1642 while each of the slave links again attempt to frame synchronize 
by each performing steps 1620, 1622, and 1624. The master again monitors the link_byte_locked chain to determine if 
all slaves are in frame sync with master at step 1644. The master repeats steps 1644 and 1646 until all slave links are 
frame synchronized, or until the master selects the last byte tap point, such as tap point 1514 of FIGURE 15A. Once 
the link_byte_locked chain is asserted, then the master waits a preselected number of additional frames at step 1648 
while each data link monitors itself for framing errors, and then the master asserts signal LSOUT at step 1650 and all 
links begin to transmit payload data over the synchronized channel at step 1652 and step 1626. The preselected 
number may be five, for example; other embodiments may wait a different number of additional frame at step 1648. In 
Table 15, the max_delta_count_c parameter specif ies the preselected number of additional frames to wait for. 
[0076] The number of data links needed in a digital system is determined by the bandwidth requirement of the data 
stream and the capabilities of the physical media and the transmitter/receiver. Referring back to FIGURE 2, other 
embodiments may have more or fewer data links. To achieve both a specified word width m and a system specified bit 
rate it may be necessary to adjust the width n of the parallel bytes transmitted on each link 1 10(i). 
[0077] Example: 16 bit interface at 75MHz on a technology with a maximum individual data link capacity of 
400Mbps. 

Channel bandwidth requirement: 1200Mbps 

[0078] Assume the number of links is i=3. This gives a per pair bandwidth of 400Mbps which is within the capabili- 
ties of the technology. However, since 16/3 is not an integer it is necessary to increase the amount of bits per link to 6, 
giving an 18 bit wide interface (i.e. two spare bits). 
[0079] The required bandwidth including this overhead is then... 

18/16 * 1200Mbps = 1350Mbps 
Per link bandwidth: 450Mbps 

[0080] This is now not viable with the chosen technology. The only option on this technology node is to increase the 
link count to four with four bit wide datapaths. There are now no spare bits and a throughput requirement for each link 
is only 300Mbps. 

[0081] The parameters listed in Table 15 are passed to the computer assisted design (CAD) software in the 
autogeneration phase of an integrated circuit which will have a communication channel 110. Based on the selected 
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parameters, an embodiment of the present invention is created using design cells from a maoro design Iforar, 



Generic 
DATAPATH^ WIDTH 


Type 
INTEGER 


Range 
4 to 10 


Description 

lhe width of the internal parallel 
datapath 


GENERATOR_POLYNOMIAL 


GENJT 




lhe generator polynomial to be used 
in the scrambler and descrambler 


~ PIPELINE J)EPTH 


INTEGER 




The number of byte stored in the 
receiver pipeline 


FRAME_COUNT_C 


INTEGER 




The frame period in number of clock 
cycles 


MAX_DELTA_COUNT_C 


INTEGER 


1 to 7 


Number of correct frames required to 
pass from PRESYNC to SYNC stat*.* 


, MAX_ALPHA_COUNT_C 


INTEGER 


1 to7 


Number of correct frames required to 
pass from SYNC to HUNT states 



Table 15 : Scalable Parameters 



be described herein techmques for gate m,n,m, 2 at.on, the details of such an embodiment will not 

Si a™»h ^ US ! Sin9 ' e Cnded Si9nalin9 or difterential si 9 nal '"S. for example 

Kof^^ 

embodiment each word of the data st^ ^S^^^? embodiments. For example, in the described 
data link. In another embodiment a Z^^Jnu^T* . ^ X ° th6data ^ wkrth of eacn 

into bytes with a bit len^3ual to S l^T ^ , WW * "** b * 35 a sin 9 le Aem «* then divided 

word bedded S^ZS.wtSCSSiSJ 2 ^TT 3 ,irSt 0f a 

of the same word may be div* ed into 1 SS schem^r ^ h ^ f "* ^ ^ 3 S6C0nd POrtion 
streams are included wrthin me scope 52 £SX£T * d ' V ' din9 3 ^ 0,dala int0 multi " e sub " 

w^dd,t^ 

Ked to^n^^ i^strawe embodiments. this Ascription is not 

skilled in the art upon reference to tnTd^n^ u 1 f mbodimen<s <* the ™ention will be apparent to persons 

Claims 
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dividing the single stream of data into a plurality of sub-streams of data; 
inserting a frame pulse periodically in each sub-stream of data; 

transmitting each sub-stream of data over a corresponding data link of the plurality of data links in a parallel 
manner; 

receiving each sub-stream of data to form a plurality of received data sub-streams, wherein a first data link has 
a first delay time that is different from a second delay time of a second data link, such that a data skew occurs 
between a first received data sub-stream and a second received data sub-stream; 
recovering a byte clock from each received data sub-stream; 

synchronizing the byte clock and frame pulse of each received data sub-stream to the byte clock and frame 

pulse of the first received data sub-stream such that the data skew is eliminated; and 

combining the plurality of received data sub-streams to form a single received stream of ordered data. 

2. A method for receiving a single stream of ordered data over a plurality of data links each having a transmitter and 
a receiver, wherein the single stream of data comprises a plurality of words each having a plurality of bits, and 
wherein the single stream of data is divided in a plurality of sub-streams of data with a frame pulse inserted peri- 
odically in each sub-stream of data, comprising the steps of: 

receiving each sub-stream of data over a corresponding data link of the plurality of data links in a parallel man- 
ner to form a plurality of received data sub-streams, wherein a first data link has a first delay time that is differ- 
ent from a second delay time of a second data link, such that a data skew occurs between a first received data 
sub-stream and a second received data sub-stream; 
recovering a byte clock from each received data sub-stream; 

synchronizing the byte clock and frame pulse of each received data sub-stream to the byte clock and frame 

pulse of the first received data sub-stream such that the data skew is eliminated; and 

combining the plurality of received data sub-streams to form a single received stream of ordered data. 

3. The method according to any preceding Claim, wherein the step of synchronizing comprises the steps of: 

forcing the byte clock of each data link to align approximately with the byte clock of the f irst data link; and 
shifting a bit tap point on each received data sub-stream so that byte boundaries of each received data sub- 
stream align with the byte clock associated with each received data sub-stream. 

4. The method according to any preceding Claim, wherein the step of synchronizing comprises the steps of: 

a) testing for frame synchronization by determining if the frame pulse of each of the received data sub-streams 
is synchronized with the frame pulse of the first received data sub-stream; and 

b) shifting a byte tap point in each of the received data sub-streams other than the first received data sub- 
stream up to a first number of times until each received data sub-stream is frame synchronized with the first 
received data sub-stream. 

5. The method according to Claim 3, further comprising the steps of: 

c) detecting a lack of frame synchronization when all received data sub-streams are not frame synchronized 
after the step of shifting a byte tap point a first number of times; 

d) shifting a byte tap point in the f irst received data sub-stream by one position after the step of detecting a lack 
of frame synchronization; and 

e) then repeating step b until each received data sub-stream is frame synchronized with the first received data 
sub-stream. 

6. The method according to Claim 4, further comprising the step of repeating the steps c, d, and e up to a second 
number of times until each received data sub-stream is frame synchronized with the first received data sub-stream. 

7. A digital system having a first node and a second node and a communication channel connected between operable 
to transfer a single stream of ordered data over the communication channel, wherein the single stream of data com- 
prises a plurality of words each having a plurality of bits, wherein the communication channel further comprises: 

circuitry for dividing the single stream of ordered data into a plurality of sub-streams of data; 

a plurality of data links each having a transmitter and a receiver, the plurality of data links operable to transmit 
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20 9. 



25 



30 



1 rf r ^ ^ overawrrespondingdata link of the plurality of data links in a parallel manner to form 

T^T ^ Sul>s,reams ' a •« data link has a first delay time that is different from a 

anriacf- h °f a second data link, such that a data skew occurs between a first received data sub-stream 
and a second received data sub-stream; s>ueam 

circuitry for inserting a frame pulse periodically in each sub-stream of data connected to each transmitter 
wherein each receiver further comprises: 

circuitry for recovering a byte clock from the received data sub-stream- and 

^ y m !T Ch r 2in9 ? 6 b/te d0Ck and <rame pulse 01 «** rece ^ da »a sub-stream to the byte clock 
and frame pulse of a preselected master one of the receivers such that the data skew is eliminated. 

The digital system according to Claim 7, wherein the circuitry for synchronizing comprises: 

circuitry for forcing the byte clock of the receiver to align approximately with the byte clock of the master 
r6C6iv6r, and 

*^<^^ ^ t0 a,i9n ^ * "» — -^eam with 

The digital system according to any of Claim 7-8, wherein the synchronizing circuitry further comprises: 

iTlSTl Cir K U i y ° Perable t0 t6St for frame s y nchroni «tion by determining if the frame pulse of the 
received data sub-stream ,s synchronized with the frame pulse of the master receiver- and 

^SlT f bytS ^° irt 0P6rable *° de ' ay ,h6 r6Ceived da,a sub-stream a 'first number of bytes in 

^ r t me *f J*"** 80 ,h3t the r6CejVed d3ta Sub ' Str6am is ,rame synchronized Jh the 
r eceived data sub-stream of the master receiver. 

10 ' td^7£Zl a T 3 COmmunication <*"™* the digital system being operable to receive a single stream of 
ordered data over the communication channel, wherein the single stream of data comprises a plurality of words 
each having a plurahty of brts, wherein the communication channel further comprises a plurality oTreceS 
according to any of Cla.ms 8-10, each receiver operable to receive a data sub-stream 
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